A/B test program

Reverse trial - in-app A/B tests

Eleven in-app messages in Customer.io campaign 50. Each has a Random Cohort Branch (50/50) inserted before it: control stays untouched, a new draft variant runs alongside. Every variant tests a distinct behavioral lever. Variants are wired as drafts - Carlos enables manually.

Campaign
Reverse Trial In-App (50)
Workspace
Carepatron 199348
Primary metric
In-app CTA click rate
Secondary metric
Paid conversion in trial (global)
Variants
11 draft, awaiting enable
Architecture
Per-message random cohort 50/50

Click any row to open the side-by-side preview with hypothesis + rationale + tracked-response-name.

Control

Variant Draft

\n \n ","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"#10101099","message_width":500,"exit_click":true}}},{"msg":2,"lever":"reciprocity","leverShort":"Reciprocity","leverFull":"Reciprocity (give-first)","day":1,"surface":"Bottom-right pop-up","hypothesis":"If Glen leads with an unclaimed-asset reframe (your included 1:1 onboarding hour) instead of asking for a discovery call, users will book because receiving a perceived gift before any ask raises engagement at day 1, when they have done little setup yet.","rationale":"Day 1. The user just got campaign 1's Welcome email and Setting up your workspace. Campaign 3 sales hasn't reached out yet. The control pop-up asks for a discovery call. This variant reframes the same Calendly slot as claiming an already-included benefit. The 1:1 onboarding hour is in the trial spec (msg 1 control names it as a feature). Reciprocity arrives before commitment-pressure does.","tracked":"RT-msg-2-variant-reciprocity","cohortId":512,"variantActionId":513,"control":{"id":281,"name":"In-app 2 - Glen Intro (Pop-up)","body":"
\"\"

Hey, I'm Glen from Carepatron! 

I'd love to hear how it's going and help you get the most from it.

","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":418,"name":"Variant - msg 2 - reciprocity","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n \n\n \n\n
\n\n\n\n \n\n\n
\n
\n \n\n \n FROM GLEN\n

\n \n Your 1:1 onboarding hour is unclaimed.\n

\n \n\n \n\n\n \n It is included with your Advanced trial. Most clinics never claim theirs.\n

\n\n\n \n Pick a time and I will walk you through the setup live, no recording, no sales pitch.\n

\n\n\n
\n \n Claim my hour\n \n
\n\n\n\n \n \n\n \n\n \n \n ","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":3,"lever":"goalgradient","leverShort":"Goal-gradient","leverFull":"Goal-gradient","day":2,"surface":"Top-center banner","hypothesis":"If the day-2 banner shows the next setup milestone in a visible 4-step ladder instead of advertising 50% off, users will engage because seeing distance-to-goal accelerates action - they are still in setup, not in pricing-mode yet.","rationale":"Day 2. By now they have seen Welcome and Setting up your workspace from campaign 1. The control bumps straight to a 50% off banner. This variant intercepts with a goal-gradient ladder: STEP 2 OF 4 - next is reminders. Pulls them deeper into activation before the trial ever has to argue on price.","tracked":"RT-msg-3-variant-goalgradient","cohortId":514,"variantActionId":515,"control":{"id":282,"name":"Banner - 50% off 6 months ","body":"

Get 50% off for your first 6 months

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":419,"name":"Variant - msg 3 - goalgradient","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n \n\n \n\n \n\n \n\n\n \n STEP 2 OF 4\n

\n\n \n Next: turn on automatic reminders.\n

\n\n
\n \n\n\n \n \n
\n \n Continue setup\n \n
\n

\n\n \n\n \n\n \n \n\n\n
\n \n\n\n \n\n \n
\n\n \n\n \n\n\n\n \n \n\n \n\n \n \n ","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":4,"lever":"specificity","leverShort":"Specificity","leverFull":"Specificity bias","day":3,"surface":"Top-center banner (no reminders branch)","hypothesis":"If the banner promises a precise, time-bounded outcome (set up automatic reminders in under five minutes) instead of an abstract 50% discount, users will click because specificity converts better than vague savings at the moment they have not yet engaged with the highest-leverage feature.","rationale":"Day 3, branch where the user did NOT view the reminders page yet. Campaign 1 has shipped the A better way to work email by now (3 day delay from welcome). Control is the same 50% off banner. This variant gets specific about the highest-leverage 5-minute task that the user has avoided. Specificity beats generic price urgency at the setup stage.","tracked":"RT-msg-4-variant-specificity","cohortId":516,"variantActionId":517,"control":{"id":283,"name":"Banner - 50% off 6 months ","body":"

Get 50% off for your first 6 months

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":420,"name":"Variant - msg 4 - specificity","body":"

Set up automatic reminders in under five minutes.

One rule, runs forever.

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":5,"lever":"ikea","leverShort":"IKEA effect","leverFull":"IKEA effect","day":3,"surface":"Bottom-right pop-up (viewed-reminders branch)","hypothesis":"If we honor the effort they invested by viewing the reminders page and ask for only the one final click instead of restarting the setup story, users will complete reminders because work already done feels more valuable to preserve.","rationale":"Day 3, branch where the user DID view reminders (proven by reminders_page_viewed event). The control runs a generic First-win pop-up. This variant explicitly honors the user's effort (you started reminders) and frames the last action as locking-in. IKEA effect compounds with sunk-cost feel: now they have invested time, finishing converts more reliably than starting fresh.","tracked":"RT-msg-5-variant-ikea","cohortId":518,"variantActionId":519,"control":{"id":280,"name":"In-app 3 - First win (Pop Up)","body":"
\"Set

Your Advanced trial includes automatic reminders. Set them up once and they run in the background, clients get notified at the right time.

","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":421,"name":"Variant - msg 5 - ikea","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n\n \n ALMOST THERE\n

\n\n\n \n You started reminders. Finish them.\n

\n\n\n \n Configure once. They run on their own from there.\n

\n\n\n \n Tap save on the rule you set up and your clients get reminded automatically.\n

\n\n\n
\n \n Finish reminders\n \n
\n\n\n\n
\n \n\n \n\n \n \n ","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":6,"lever":"socialproof","leverShort":"Social proof","leverFull":"Social proof","day":4,"surface":"Top-center banner","hypothesis":"If the day-4 banner cites the verified scale of Carepatron's clinical user base (100,000+ practitioners) instead of the 50% offer, users will respond to the social signal that they are joining a real choice clinicians already make.","rationale":"Day 4. They have seen the You're in good company email from campaign 1 already (which uses the same social-proof angle). The control bumps with yet another 50% banner - that pricing message is now repeating. This variant uses a verifiable social-proof fact at the right surface, complementing - not duplicating - the email. Banner cadence keeps it light.","tracked":"RT-msg-6-variant-socialproof","cohortId":520,"variantActionId":521,"control":{"id":285,"name":"Banner - 50% off 6 months ","body":"

Get 50% off for your first 6 months

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":422,"name":"Variant - msg 6 - socialproof","body":"

100,000+ practitioners trust Carepatron with their workspace.

Pick the plan that fits the way you already work.

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":7,"lever":"peakend","leverShort":"Peak-end","leverFull":"Peak-end framing","day":7,"surface":"Modal takeover (light)","hypothesis":"If we ask the user to recall the one moment Advanced earned its place this week instead of reciting feature loss, users will engage with the decision because self-supplied recall creates ownership of the value, where the control's feature-list approach can read like guilt-trip.","rationale":"Day 7. The control says you'll lose Reminders/Calendar sync/Custom branding/unlimited storage - which is exactly what campaign 1's Free trial ending in 10 days email will recite a few days later. This variant uses a peak-end prompt: name the moment that earned it. Self-supplied evidence is harder to argue with than someone telling you what you'll lose, and it complements (not duplicates) the loss-aversion email arriving soon.","tracked":"RT-msg-7-variant-peakend","cohortId":522,"variantActionId":523,"control":{"id":284,"name":"In-app - 7 Days Left (Modal Takeover)","body":"
\"Get

Loving Advanced?

Your Advanced plan ends next week and then you move to Free.

That means losing:

  • Reminders
  • Calendar sync
  • Custom branding
  • and unlimited storage

If Advanced fits, now is the time.

","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","overlay_color":"#CCCCCC80","message_width":1000,"exit_click":true}},"variant":{"id":423,"name":"Variant - msg 7 - peakend","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n\n \n HALFWAY THROUGH\n

\n\n\n\n What has Advanced earned its place doing?\n\n\n \n The clinics that stay on Advanced usually have one moment that earns the upgrade. The calendar sync that ended the inbox tag-war. The reminders that quietened the late cancels. The workflow that put a paperwork hour back on the calendar.\n

\n\n\n \n You have a week left to find yours. If you have already found it, lock it in.\n

\n\n\n
\n \n Keep Advanced\n \n
\n\n\n\n
\n \n\n \n\n \n \n ","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"#10101099","message_width":500,"exit_click":true}}},{"msg":8,"lever":"authority","leverShort":"Authority","leverFull":"Authority (compliance)","day":11,"surface":"Modal takeover (dark, Stripe-disabled branch)","hypothesis":"If we frame the call as a payments-compliance expert helping them past PCI/security uncertainty instead of a generic 15-minute setup call, users will book because the authority signal addresses the regulatory anxiety likely behind why Stripe is still off.","rationale":"Day 11, branch fires only if payments_stripe_enabled = false. They have seen one or two trial-countdown emails. The control opens with the $1,000 fee credit. This variant leads with PCI compliance + audit-ready framing because clinicians not yet on Stripe are usually stuck on the compliance read, not the price. Authority addresses the actual blocker.","tracked":"RT-msg-8-variant-authority","cohortId":524,"variantActionId":525,"control":{"id":317,"name":"In-app 1 DARK - Modal Takeover (Trial Start)","body":"

FREE $1,000 IN PAYMENT PROCESSING

15 minutes to start taking card payments

Setting up online payments in Carepatron is one of the big unlocks.

Book a 15-min call with our payments team and we'll get you live. Your first $1,000 in processing fees is on us.

","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"#00000080","message_width":500,"exit_click":true}},"variant":{"id":424,"name":"Variant - msg 8 - authority","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n\n \n PAYMENTS, AUDIT READY\n

\n\n\n\n PCI compliant card payments. Live in fifteen minutes.\n\n\n \n Stripe via Carepatron is PCI compliant out of the box. Every transaction, refund, and payout sits on one dashboard your accountant can read.\n

\n\n\n \n Book fifteen minutes with our payments specialist. They handle the compliance pieces so you do not have to read Stripe's terms cover to cover.\n

\n\n\n \n Your first $1,000 in processing fees stays on us.\n

\n\n\n
\n \n Book the call\n \n
\n\n\n\n
\n \n\n \n\n \n \n ","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"#10101099","message_width":500,"exit_click":true}}},{"msg":9,"lever":"commitment","leverShort":"Commitment","leverFull":"Commitment / consistency","day":12,"surface":"Bottom-right pop-up","hypothesis":"If we reflect the user's own 12-day investment back to them instead of warning about feature loss, users will choose to keep Advanced because consistency pressure (you have already been an Advanced user for nearly two weeks) is harder to argue with than abstract loss.","rationale":"Day 12. Campaign 1 has already sent Free trial ends in 10 days and Free trial ends in less than a week. Loss-framing is saturated by now. This variant flips to commitment/consistency: name the user's investment back to them. Eyebrow says DAY 12 OF 14. The argument is their own past behavior.","tracked":"RT-msg-9-variant-commitment","cohortId":526,"variantActionId":527,"control":{"id":286,"name":"In-app 2 - 2 Days left (Pop Up)","body":"
\"Set

You'll miss out on telehealth, custom templates, and automated reminders.

Upgrade to keep everything working the way it does now.

","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":425,"name":"Variant - msg 9 - commitment","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n\n \n DAY 12 OF 14\n

\n\n\n \n You have run on Advanced for almost two weeks.\n

\n\n\n \n That is enough signal. If Advanced has fit your week, lock it in.\n

\n\n\n \n Your reminders, calendar sync, and workflows keep running with no break.\n

\n\n\n
\n \n Keep Advanced on\n \n
\n\n\n\n
\n \n\n \n\n \n \n ","display":{"mode":"overlay","value":"x-gist-floating-bottom-right","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":10,"lever":"freshstart","leverShort":"Fresh-start","leverFull":"Fresh-start effect","day":13,"surface":"Top-center banner (purple)","hypothesis":"If the day-13 banner frames the upgrade as choosing how next week starts rather than a discount, users will click because the temporal landmark of a new week is a stronger commitment trigger than a price.","rationale":"Day 13, one day before the trial closes. Campaign 1 ships Your free trial ends tomorrow (with $1 offer + 50% off variants) tonight or tomorrow morning. The control runs ANOTHER 50% banner. This variant instead invokes a temporal-landmark: choose how next week starts. Fresh-start primes commitment more reliably than repeat-discount framing at this stage.","tracked":"RT-msg-10-variant-freshstart","cohortId":528,"variantActionId":529,"control":{"id":288,"name":"Banner - 50% off 6 months ","body":"

Get 50% off for your first 6 months

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","overlay_color":"","message_width":0,"exit_click":false}},"variant":{"id":426,"name":"Variant - msg 10 - freshstart","body":"

Choose how your next week starts.

Pick a plan before the trial closes tomorrow.

","display":{"mode":"overlay","value":"x-gist-floating-top","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"","message_width":0,"exit_click":true}}},{"msg":11,"lever":"scarcity","leverShort":"Honest scarcity","leverFull":"Honest scarcity","day":14,"surface":"Modal takeover (dark)","hypothesis":"If the final-day modal removes the 'remind me tomorrow' snooze and anchors the decision to the actual time the plan expires today, users will decide now because honest scarcity converts better when the escape hatch is removed.","rationale":"Day 14 - trial actually ends. Campaign 1's final-reminder email lands today too. The control modal includes a 'Remind me tomorrow' secondary CTA, which leaks urgency - tomorrow does not exist for this trial. The variant uses real scarcity (today is the boundary) and removes the snooze. Same loss framing as the email, but the in-app surface owns the time-of-decision anchor.","tracked":"RT-msg-11-variant-scarcity","cohortId":530,"variantActionId":531,"control":{"id":287,"name":"In-app - Final Day (Modal Takeover)","body":"
\"Get

Your Advanced Plan trial ends today

Today you move from Advanced to Free. You'll lose reminders, calendar sync, branding, workflows, and premium support.

Upgrade now and your Advanced plan stays as it is.

","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","overlay_color":"#CCCCCC80","message_width":1000,"exit_click":true}},"variant":{"id":427,"name":"Variant - msg 11 - scarcity","body":"\n \n \n \n \n \n \n \n \n \n \n \n
\n\n\n \n\n \n\n\n\n\n
\n \n\n\n \n\n \n
\n\n\n \n TODAY\n

\n\n\n\n Your Advanced trial expires today.\n\n\n \n By morning your workspace drops to Free. Reminders, calendar sync, custom branding, and workflows turn off automatically.\n

\n\n\n \n If Advanced fit your fortnight, lock it in before the day closes.\n

\n\n\n
\n \n Keep Advanced\n \n
\n\n\n\n
\n \n\n \n\n \n \n ","display":{"mode":"modal","value":"center","tooltip_target":"","tooltip_position":"","tooltip_arrow_color":"","overlay_color":"#10101099","message_width":500,"exit_click":true}}}]; function leverShort(d) { return d.leverShort; } function renderGrid() { const grid = document.getElementById('grid'); grid.innerHTML = DATA.map((d, idx) => `
${d.msg}Day ${d.day}
${d.leverShort}
${d.leverFull}. ${escapeHtml(d.hypothesis.replace(/\.\s.+$/, '.'))}
Draft
CTR pending
paid conv pending
`).join(''); } function escapeHtml(s) { return s.replace(/&/g, '&').replace(//g, '>'); } function openDetail(idx) { const d = DATA[idx]; document.getElementById('d-title').innerHTML = `Msg ${d.msg}: ${d.leverFull}`; document.getElementById('d-sub').textContent = `Day ${d.day} of 14 - ${d.surface}`; document.getElementById('d-hyp').textContent = d.hypothesis; document.getElementById('d-rat').textContent = d.rationale; document.getElementById('d-tracked').textContent = d.tracked; document.getElementById('d-cohort').textContent = d.cohortId; document.getElementById('d-vid').textContent = d.variantActionId; document.getElementById('d-vtid').textContent = d.variant.id; document.getElementById('d-control-footer').textContent = `tpl ${d.control.id} - ${d.control.display.mode}`; document.getElementById('d-variant-footer').textContent = `tpl ${d.variant.id} - ${d.variant.display.mode}`; document.getElementById('d-control-frame').srcdoc = d.control.body; document.getElementById('d-variant-frame').srcdoc = d.variant.body; document.getElementById('overlay').classList.add('open'); document.body.style.overflow = 'hidden'; } function closeOverlay(ev) { if (ev && ev.target.id !== 'overlay') return; document.getElementById('overlay').classList.remove('open'); document.body.style.overflow = ''; } document.addEventListener('keydown', e => { if (e.key === 'Escape') closeOverlay(); }); renderGrid();